Image processing by determining combined image distances using dual path analyses

ABSTRACT

Image analysis for comparison utilizes a combined image distance determined between the one or more captured images and the one or more reference images from a dual path analysis. The dual path analysis can comprise a first image analysis based on histograms of image colors and a second image analysis based a one-dimensional power spectral density of a black and white instance of the one or more captured images and the one or more reference images. In one case, the one-dimensional power spectral density derived by integrating a Discrete Fourier Transform of a two-dimensional image over annular rings of substantially equal area starting from the center of the two-dimensional image.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation of U.S. application Ser. No.15/083,140, filed on Mar. 28, 2016 by Prashant Choudhary and entitledAutonomous Cooking Device to Prepare Food from a Recipe File and Methodof Creating Recipe Files, which in turn claims priority to, U.S.Application No. 62/138,985, filed on Mar. 27, 2015 by PrashantChoudhary, et al. and entitled Autonomous Cooking Device to Prepare Foodfrom a Recipe File and Method for Creating Recipe Files, the content ofboth being incorporated herein by reference in their entirety.

FIELD OF THE DISCLOSURE

The invention relates generally to autonomous cooking devices, and morespecifically, to image processing by determining combined imagedistances using dual path analyses.

BACKGROUND OF DISCLOSURE

Recipes are a general ideal set of cooking instructions for manual foodpreparation. Some recipes are written down on index cards for personalreference, or published in a cook book or even online for sharing. Evenwith the instructions, a novice cook can find it difficult, or at leasttedious, to replicate the cooking process of a professional cook. It canalso be difficult to consistently recreate personal recipes.

Existing cooking devices have only a basic level of automation. Forexample, a microwave oven cooks according to a programmed time at aprogrammed power level. A coffee machine can automatically brew a pot ofcoffee at a programmed time using water and coffee beans. However, noneof these machines are able to operate in an intelligent manner. Forexample, these machines are unable to adapt to actual conditions duringpreparation. In another example, none of these machines are able tolearn new recipes.

It is desirable to overcome these shortcomings. Therefore, what isneeded is a robust autonomous cooking device to automatically preparefood. The device should use computer vision and machine learning tolearn new recipes, and the device should adapt to sensor feedback inreal-time during preparation. Furthermore, the device should benetworked for retrieving and sharing or selling recipe files within anonline community.

SUMMARY OF THE DISCLOSURE

To meet the above-described needs, image processing is performed bydetermining combined image distances using dual path analyses.

In one embodiment, an image sensor captures one or more images of asubject for comparison against one or more reference images. A controlunit comprising an image analyses module to compare one or more capturedimages against one or more reference images stored in the storagedevice.

In another embodiment, a combined image distance is determined betweenthe one or more captured images and the one or more reference imagesfrom a dual path analysis. The dual path analysis can comprise a firstimage analysis based on histograms of image colors and a second imageanalysis based a one-dimensional power spectral density of a black andwhite instance of the one or more captured images and the one or morereference images. In one case, the one-dimensional power spectraldensity derived by integrating a Discrete Fourier Transform of atwo-dimensional image over annular rings of substantially equal areastarting from the center of the two-dimensional image.

In still another embodiment, the image analysis module computes thecombined image distance measure by combining an image distance measuregenerated from the color histograms data with an image distance measuregenerated from the one dimensional power spectral density comparison,and in one case applies adjustable weights for each of the first andsecond paths. Next, the control unit takes an action to affect externalto the image processing unit based on the combined image distance fromthe image comparison.

Advantageously, image analysis for comparison is more accurate.

BRIEF DESCRIPTION OF THE DRAWINGS

In the following drawings, like reference numbers are used to refer tolike elements. Although the following figures depict various examples ofthe invention, the invention is not limited to the examples depicted inthe figures.

FIG. 1 is a perspective view illustrating an autonomous cooking deviceutilizing recipe files, according to one embodiment.

FIG. 2 is a bottom perspective view illustrating a mixing blade assemblyof the autonomous cooking device of FIG. 1, according to one embodiment.

FIG. 3 is a perspective view illustrating a lid assembly of theautonomous cooking device of FIG. 1, according to one embodiment.

FIG. 4 is an exploded view illustrating an ingredient dispensingplatform of the autonomous cooking device of FIG. 1, according to oneembodiment.

FIG. 5 is a perspective view illustrating a cooking unit with threedegrees of freedom for mixing in the autonomous cooking device of FIG.1, according to one embodiment.

FIG. 6 is a block diagram of a finite state machine for electronics ofthe autonomous cooking device, according to one embodiment.

FIG. 7 is a network architecture including the autonomous cooking devicefor sharing recipe files, according to one embodiment.

FIG. 8 is a flow diagram illustrating a method for food preparation inthe autonomous cooking device, according to one embodiment.

FIG. 9 is a flow diagram illustrating a step of training the autonomouscooking device in the method of claim 8, according to one embodiment.

FIG. 10 is a flow diagram illustrating a step of automatically preparingdishes with the autonomous cooking device in the method of claim 8,according to one embodiment.

FIG. 11 is a flow diagram illustrating a step of computing a measure ofcloseness between two images, in determining an end of a cooking phasein the autonomous cooking device in the automatic preparation step ofclaim 9, according to one embodiment.

FIG. 12 is a diagram of a table illustrating a recipe file, according toone embodiment.

FIG. 13 is a diagram illustrating image processing for texture analysissteps 1110, 1114 of FIG. 11, according to one embodiment.

DETAILED DESCRIPTION OF THE DISCLOSURE

Autonomous cooking devices, and methods operating therein, aredescribed. At a high-level, the device can automatically determineoptimum cooking time (i.e., remaining cooking time) for different phaseof cooking for various types of food cuisines such as American, Indian,Chinese, or others. Cooking tasks include boiling eggs, brewing tea orcoffee, making yogurt, preparing curries and the like. The design isintuitive and optimized for mass production. Aspects of intelligence areeasily trainable and components require minimal cleaning. A compact formfactor helps for easy storage and deployment.

Many other aspects of the system are possible within the spirit of thepresent invention, as will be apparent to one of ordinary skill in theart. The techniques can be implemented in any suitable computer systemhaving network access using a combination of computer software and/orcomputer hardware. Accordingly, the following details are non-limitingand are set for only for the purpose of illustration of preferredembodiments which can be varied within the spirit of the currentinvention by those of ordinary skill in the art.

I. Components of the Cooking Device

FIG. 1 is a perspective view of an autonomous cooking device 100,according to one embodiment. The cooking device 100 generally includesan outer shell 110, a cooking unit 120, an ingredient container 130, anda control unit 140.

A. Outer Shell

The outer shell 110 covers the internal components which are affixed toa base. In FIG. 1, a door is opened or a panel is removed for access tothe internal components, including the cooking unit 120 and a mixingblade shaft 105 and rotating platform 115 with a heating element. A lid125 rests on top of side walls of the outer shell 110 and an ingredientscontainer 130 is also integrated with the shell. The specificconfiguration of FIG. 1 is merely one example of numerous possiblevariations.

The digital camera is one type of cooking sensor that provides feedbackduring the cooking process. A camera hole 125 can be bored in a lid 200of the outer shell 110 in one embodiment, or on the side or otherappropriate location in other embodiments. The outer shell 110 can bemade of an opaque material which can block ambient lighting in order toprotect the integrity of internal images captured by the digital cameraduring cooking. Some embodiments of the digital camera have auto-focusand calibration capabilities. An LED light provides a constantflicker-less source of light for capturing images. Captured images canbe passed to an image processing unit as input to adaptations made onthe fly. The captured images can also show levels of or identifyingredients. An air nozzle 135 or other device self-cleans a cameralens, for example, to remove food debris, steam or condensation.

In one embodiment, a thermal imaging camera is also used to sense howwell the ingredients are cooked from inside which might not be obviousfrom visual images in certain situations. For example, an algorithmusing color histograms and power spectral densities can determine adifference between cooking food image and reference photos of fullycooked food (or of individual cooking phases for a cooked dish).Additional sensors that can be optionally connected to the outer shell110 are described below.

The exhaust fan can also be attached to a lid to remove smoke and vaporsfrom inside of the device. A filter can be periodically cleaned.

A touch screen user interface or other display device on the outer shellshows information and receives user input. Examples of displayedinformation and input mechanisms include ingredient levels and status,temperature readings and settings, cooking progress, video cameraoutput, user instructions, recipe information, start and stop buttons,control knobs, time and the like. Input/output interface such a USB canalso be included on the outer shell.

Other computer circuitry such as the processor and software can bedeployed in the outer shell along with shielding for protection from theheat, fumes, moisture and other dangers to electronics.

B. Ingredient Container

The ingredients container 130 holds meal ingredients such as water,milk, flour, eggs, meat, vegetables, cooking oil, herbs, and seasonings.There are several equal-sized containers (e.g., 2 to 10 containers), orvarying sized containers formed by splitting a standard container intosub-containers. The containers can be different shapes A total volume ofthe dispensers is preferably greater than the volume of the cookingunit. The containers are easily removed for loading, cleaning, and to bemanipulated by the cooking device. Containers can be removedindividually or the whole container unit can be removed for easycleaning. A shutter can cover the containers in order to preserve andprotect ingredients from cooking fumes and smoke. In some embodiments,disposable containers (e.g., available in a grocery store or by mail)can be used. In other embodiment, a cooling element refrigerates one ormore of the containers.

As directed by the control unit 140, the ingredients container 130 canuse electrical and mechanical components to autonomously add contents tothe cooking unit 120 at a particular time. A weight sensor or volumesensor can manage adding, for example, half a container of flour at afirst time and the other half of the container at a second time.

The ingredients container 130 shown in FIG. 1 is disposed near thecooking unit 120. A container 410 rests on a container base 405 whichconnects to a container platform 420. A motor 425 turns a bearing 430connected to the container platform 420. A solenoid 440 pulls down aspring-loaded ferrous container lock 450 to release container base 405.The controller unit 140, having knowledge of which ingredients are inwhich containers, can rotate the ingredients container 130 similar to acarousel until the appropriate container reaches a position fordispensing. A solenoid 440 pulls container lock down.

In one embodiment, a water supply is connected for dispensing into thecooking process. In another embodiment a water dispenser is integratedwith the device.

C. Cooking Unit

A detail of the cooking unit 120 with the mixing blade 105 and therotating platform 115 is shown in FIG. 5. An underside of the lid 125with a connection to the mixing blade shaft 105 is shown in FIG. 2. Amixing blade mount 145 affixes the mixing blade shaft 105 to a mixingwheel 155. Other components of the cooking unit 120 (not shown inpictures) include a heating element and an optional cooling unit. Theheating element could be an inductive coil, a resistive heating coil orany other means.

As shown in FIG. 3, a removable cooking lid 330 is attached to a heavylid holder base 320 which can be moved up or down using three steelcables 350. A cooking lid slides over the mixing blade shaft 105. Amixing blade motor 340 turns a mixing blade rotator 310 connected to themixing blade shaft 105 which turns a spring-loaded mixing blade 360, inthe current embodiment.

The mixing blade 360 can be a general-purpose mixing blade that worksacross different phases of cooking (e.g., similar blade to Kitchen-Aidstand mixer). The mixing blade 360 is strong enough to mix viscouscookie dough as well as delicate ones. The mixing blade 360 is removablefor cleaning or switching out different sizes. In some embodiments, morethan one mixing blade is provided. A stepper motor or DC servo motor 340drives the mixing blade 360 using electric power to spin a mixing bladerotator 310.

In one embodiment, the mixing blade 360 can also move vertically up ordown to allow for mashing of ingredients. The cooking unit 120 can beindependently rotated in addition to rotation and up and down traversalof the mixing blade 360 to give more degrees of freedom in mixingcompared to a stand-mixer mechanism. Using camera feedback or othersensors, the motor can rotate the main cooking utensil to necessarypositions. A dual motor mixing mechanism can rotate the blades and thecooking platform. The mixing blade shaft 105 is mounted to the lid 125in FIG. 1 but can alternatively be attached to a base. A paddle orscooper serves as the main cooking utensil in this embodiment. Parts canbe interchangeable in order to substitute the mixing blade with a mixeror other tool. Additional variations are possible.

In one embodiment, a cooling unit refrigerates dishes after preparation.The cooling unit can be combined with or independent of ingredientrefrigeration. The cooling unit can also reduce temperatures for variousphases of cooking.

D. Control Unit

The control unit 140 includes and cooperates with several components asthe brains of the cooking device 110, an example of which is shown inthe block diagram of FIG. 6. Generally, a processor, a memory, aninput/output device, and various electronic sensors automate foodpreparation. The control unit 140 can be isolated from the harsh cookingenvironment in order to protect sensitive electronics by usinginsulation, shields, or the outer shell 110. Based on input from sensors605A-695E, a finite state machine and control logic 640 determinesoutputs for controller/driver circuitry 650 in adjusting components695A-695F.

A touch display and user interface 620 can receive instructions by auser performing manual steps during a training phase. For example, whiletraining for preparation of brownies, contents of the ingredientcontainers 130 can be found in a local or networked database (e.g.,recipe and ingredient database 610) and associated with the recipe filefor brownies. Besides preparation instructions, some embodiments alsosave nutritional information and images to the recipe file. Theinput/output device can also display real-time information from sensors(e.g., temperature, predicted finish time, and other data).

Sensors 605A-605E provide real-time feedback about current conditions ofthe food being prepared. This allows for sensor levels to be recordedduring training, and to make sure the food being prepared according to arecipe file is cooking as expected. For example, a temperature sensor605C measures a temperature of meat being cooked or soup being prepared.In another example, the thermal imaging camera 605A can be used toensure food is perfectly cooked and not under-cooked or over-cooked,using infrared technology. In yet another example, a video cameracoupled to video recognition capabilities as another form of monitoring.The video camera can also stream to a remote online resource forprocessing. In still another example, one or more weight sensors 605Daids administration of ingredients as well as serving to monitor cooking(e.g., monitoring change in weight as an indication of cookingprogress). A position sensor 605E and many other types of sensors arepossible. The user interface can be used to view sensor parameters, andfor manual input or updates to sensor data.

In more detail, a visual imaging camera 605B is one type of cookingsensor that provides feedback during the cooking process. The visualimaging camera 605B can be attached to a lid of the outer shell 110 inone embodiment, or on the side or other appropriate location in otherembodiments. The outer shell 110 can be made of an opaque material whichcan block ambient lighting in order to protect the integrity of internalimages captured by the digital camera 605A during cooking. Someembodiments of the digital camera have auto-focus and calibrationcapabilities. An LED light provides a constant flicker-less source oflight for capturing images. Captured images can be passed to an imageprocessing unit as input to calculate remaining cooking time bycomparing current images of food to reference images, as describedfurther below with respect to FIG. 11. The captured images can also showlevels of or identify ingredients. An air nozzle or other deviceself-cleans a camera lens, for example, to remove food debris, steam orcondensation.

A recipe file 1200 is shown in the table of FIG. 12. The table showssensor data from a camera, a thermometer, and a scale. Reference imagescan be stored within the recipe file 1200, in memory, or online.Real-time data can be plotted on the table to identify a correspondingtime. In some cases, such as depending on a type of dish being prepared,the camera data is weighted more than the scale data, when plotting to aparticular row on the table. Many implementation-specific algorithms arepossible. Ingredients along with amounts are also described in therecipe file 1200. Instructions can be grouped by the different cookingphases I-IV 1202, 1204, 1206, 1208 because settings may need automaticadjustment in between phases. Rows of sensor data are shown forvimageData or visual image sensor data 1212, timageData or thermal imagesensor data 1214, Wt_ or weight sensor data 1216 and Temp_ ortemperature sensor data 1218. In other embodiments, commands andcomputer instructions are also included in the receipt file 1200.

The central processing unit can comprise a standard PC processor, amobile processor, or a specialized processor or ASIC. In one embodiment,a connected mobile device or online server performs processing used inthe cooking process. An image processing unit 630 implemented inhardware and/or software is available to the finite state machine andcontrol logic 640 captures and processes data from multiple sensors andcreates a recipe file. During autonomous cooking process, the data inthe recipe file is used as a reference to compare with current sensordata and decide optimal cooking time to get same results as duringtraining process. An optimal cooking time module 660 determines how muchcooking time remains using output from the image processing unit 630which determines a difference between real-time and reference images.Additional data inputs can optionally take into consideration the othersensor data versus the other reference sensor data concerning weight,temperature, and the like. The remaining cooking time can be inreference to individual phases, or in reference to the dish as a whole.Advantageously, individual cooking phases can be automatically detectedand additional ingredients added, or cooking temperature be changed asexamples automated responsive actions using components 695A-695F.

The memory device can store multiple recipe files and source codeexecutable during operation of the cooking device 110, as describedherein.

E. Network Interface

External resources can be accessed by the cooking device as shown in theexample of FIG. 7. A network interface such as a MAC card, IEEE 802.11interface, or cellular network interface connects the cooking device toa central server 710, search engines, and other external resources. Amobile app 720A, 720B can be used as a proxy for network communicationsand then connected in communication with the cooking device for datatransfer.

In one embodiment, a user generates recipe files at a recipe creatormachine 730A which can be the cooking device 100 or some other devicesuch as a personal computer or smart phone. Manual steps can be actualsteps that are recorded while preparing a food dish, or can beinstructional steps that are entered to a tablet computer, in order togenerate recipe files. The uploaded recipe file can be stored privatelyat a user profile (e.g., Dropbox account), shared privately, sold to thepublic, or the like. Other users, or the same user at a differentcooking device can download recipe files to a recipe user machine 730B.

In another embodiment, food preparation can be controlled remotelythrough network connections and local cameras and other sensors. Thecentral server 710 or a user can get remote feedback and send commandsin response, to affect food preparation.

III. Methods of Operation in the Cooking Device

One high-level method 800 of operation is illustrated in the flowdiagram of FIG. 8. The cooking device is trained by creating recipefiles which contain processed sensory data captured during training forvarious phases of cooking a recipe (step 810). In one embodiment, therecipe files are uploaded to an online community for sharing. Trainingdetails are discussed further in reference to FIG. 9 below.

When ready to make a particular dish, an appropriate recipe file isselected and loaded into the cooking device (step 820). In anembodiment, the recipe file can be downloaded from the online community,for example, responsive to a database query. A user can be guided toload ingredients (step 830). Visual feedback, or audio instructions canbe given to a user, and sensors can verify when ingredients are loaded,and make sure adequate amounts are supplied.

Finally, the cooking device automatically prepares the dish with loadedingredients, according to phases of the recipe file (step 840). Sensorsare used to monitor cooking and feedback a state such as how muchcooking time remains or when a new phase of cooking should begin, as setforth in FIG. 10.

FIG. 9 shows one example of the training step 810 from FIG. 8. A userenters recipe ingredient information (step 902). The ingredients aregrouped in to batches (step 904) and are ordered for loading into acooking device (step 906). Feedback from the cooking device displays howto best allocate ingredients for loading into containers of aningredient dispenser unit (steps 908 and 910). For some recipes, thecooking device starts with preheating a cooking unit (step 912).

The process of cooking a recipe can be divided into phases where a phaseis a period of cooking with same set of ingredients and no change in setcontrol knobs. For example, boil milk at 180 F while constantly stirringtill user adds tea leaves is one phase of tea recipe. In anotherexample, cooking onions at medium stove settings till they arecaramelized is one phase. During training process, a phase is terminatedby a user action like change in stove setting or adding the next batchof ingredients etc. End of current cooking phase automatically startsthe next phase of cooking. During training process, as shown in FIG. 9,user controls the device by monitoring through a display and the cookingdevice continues storing sensor data from including temperature, weightand images (loop with step 914, 916, 924, 926) until the user determinesthat a phase is complete or that the dish is fully prepared (step 918).Before commencing the next phase, the user adds the next batch ofingredients and adjusts settings as needed (step 920). The cookingdevice continues recording sensor data for the phase (step 922). Oncethe phases are complete (step 918), the cooking device extracts usefulstatistics for time, temperature and weight for storage in the recipefile (step 928).

FIG. 10 shows one example of the automatic cooking step 840 from FIG. 8.A number of phases is set (step 1002). Sensory data from one or moresensors is compared with recorded sensory data from (step 1004) todetermine when a phase has ended (step 1006). More specifically, cookingprocess in encapsulated into a recipe file by breaking up the processinto phases. For example, tea preparation can be represented in fivephases: phase 1 to pre-heat utensils for a fixed amount of time and addswater to the cooking utensil; phase 2 boils water and adds additionalingredients such as tea, cardamoms and ginger; phase 3 boils theingredients for a certain amount of time and then adds milk; phase 4continues boiling; and phase continues to cook at a lower heat.Different instructions from the recipe files can be executed withindifferent phases, or specified for the beginning or end of a phase.Other instructions include mixing, opening or closing lids, stirring,and the like.

As such, one main task of the device during autonomous cooking mode isto replicate decision-making ability of an expert cook and decide when aparticular phase is completed. This is accomplished in a two-stepprocess. The first step is a coarse decision step (step 1002, 1004,1006, 1008, 1024, 1022) where the current sensory data is compared withrecorded sensory data at all time points of ongoing cooking phase.Coarse decision step ends when current sensory data is closest to thelast time point of ongoing cooking phase at-least K times. To furtherimprove the accuracy of cooking timing, a second fine decision step(1010, 1012, 1014, 1016) follows the coarse decision step. In finedecision step sensory data from last M time points is correlated with amoving window of M time points in recorded data to determine the optimaltime T_finish to end the current cooking phase. Subsequently, ongoingcooking phase proceeds for a period of T_finish seconds before moving tonext phase of cooking.

In one embodiment, cooking errors for a phase are recognized byanalyzing sensor data (step 1024) and in some cases cooking may be ended(step 1022). For e.g., if the statistical distance between current imageat start of a cooking phase and recorded image at start of the samephase is more than a pre-defined threshold, the ingredients are likelyvery different from recommended ingredients for the recipe and error isflagged to the user. In another example, if the weight of theingredients in cooking unit is significantly off the weight in recordedrecipe file, error is flagged to the user.

FIG. 11 shows one example of the sensory comparison step of FIG. 10, forimages. A current thermal or visual is received (step 1102). The imageis resized to N×N (step 1104) and formed into M sub-images of N×N size(step 1106). In other words, a 500×500 pixel image is reduced forprocessing efficiency and processed as a 100×100 image in step 1102. The500×500 image is also broken into 25 sub-images of 100×100 in step 1104.As a result, the algorithm considers the overall image comparison aswell as sub-image comparisons. In some embodiments, sub-images can beunequally weighted to reduce the influence of background colors and toincrease the influence of the middle of a turkey, for example.

Next, the dual paths of image comparison (i.e., steps 1104 and 1106)each perform two separate analyses: a first analysis is based on colorsthrough a color histogram; and a second analysis is a texture analysisthat is devoid of color and instead relying on a PSD (power spectraldensity). The PSD comparison is directed towards a texture of the food.For example, frozen chicken has a texture that is distinct from thawedchicken, as well as from fully cooked chicken. Therefore, textureanalysis can indicate the progress of a particular phase.

The first of dual path analyses (i.e., histogram analysis) is based oncolors. In more detail, the images are extracted into image histogramsfor different colors (steps 1108, 1112) so that a statistical distancecan be computed with a histogram vector of at least one reference image(steps 1116, 1120).

The second of dual path analyses (i.e., texture analysis) is preferablybased on a black and white image rather than colors. As shown in FIG.13, a black and white image 1310 showing texture of either food beingprepared or of a reference food in preparation, is represented as:

I(r,c)

The square-shaped, black and white image 1310 is integrated overconcentric circles of equal area 1320, 1330 to calculate a PSD (steps1110, 1114) and are represented as:

Re{I _(Dft)(u,v)}

for 1320; and

Im{I _(DFT)(u,v)}

for 1330In turn, a statistical distance is computed with respect to a PSD vectorof at least one reference image (steps 1118, 1122). As a result, aone-dimensional PSD is computed for a two-dimensional image. Eachelement of one-dimensional PSD is obtained by integrating a square ofDiscrete Fourier Transform of the two-dimensional image over annularrings of equal area starting from the center, using, for example, thefollowing formula:

${PSD}_{i} = {\sum\limits_{{({u,v})} \in d_{i}}\; {\langle{\left( {{Re}\left( {I_{DFT}\left( {u,v} \right)} \right\}} \right)^{2} + \left( {{Im}\left\{ {I_{DFT}\left( {u,v} \right)} \right\}} \right)^{2}}\rangle}}$

This method of computing PSD allows comparison of texture for two imagesindependent of orientation (e.g., a real-time image can be rotated 40degrees within its frame and relative to a reference image). Otherembodiments can have one, two, or more paths for analyses, using thehistogram and texture analysis of the present example, or other types ofimage analysis.

In combination with weights W1, W2, W3 and W4, a statistical measure ofthe distance between the current image and the reference image isdetermined (step 1124) for determining a state of a phase or cookingprocess. The value of the weights can be fixed across all recipes, or itcan be determined for a phase of recipe during post-processing of recipedata. For example, if for one cooking phase, the end of phase is betterindicated with color histograms, then W1 and W3 can have higher weightthan W2 and W4. Weightings can be adjusted per recipe file, per type ofdish, be manually set, or be otherwise adjusted as appropriate. Thus,the two separate analyses for each of the dual paths of imagecomparison, can be individually weighted. Further, the resized image ofstep 1104 and the sub images of step 1106 can be weighted separately,between each other and between the dual paths of analysis.

As will be understood by those familiar with the art, the invention maybe embodied in other specific forms without departing from the spirit oressential characteristics thereof. Likewise, the particular naming anddivision of the portions, modules, agents, managers, components,functions, procedures, actions, layers, features, attributes,methodologies, data structures and other aspects are not mandatory orsignificant, and the mechanisms that implement the invention or itsfeatures may have different names, divisions and/or formats. Theforegoing description, for purpose of explanation, has been describedwith reference to specific embodiments. However, the illustrativediscussions above are not intended to be exhaustive or limiting to theprecise forms disclosed. Many modifications and variations are possiblein view of the above teachings. The embodiments were chosen anddescribed in order to best explain relevant principles and theirpractical applications, to thereby enable others skilled in the art tobest utilize various embodiments with or without various modificationsas may be suited to the particular use contemplated.

I claim:
 1. An image processing unit, at least partially implemented inhardware, for improving comparison of images by determining combinedimage distance from dual path analyses, the image processing unitcomprising: an image sensor to capture one or more images of a subjectfor comparison against one or more reference images; and a control unitelectrically coupled to the image sensor and to a storage device andcomprising a processor and a memory device, the control unit comprisingan image analyses module to compare one or more captured images againstone or more reference images stored in the storage device, wherein theprocessor performs the comparison by determining a combined imagedistance between the one ore more captured images and the one or morereference images from a dual path analysis, the dual path analysiscomprising a first image analysis based on histograms of image colorsand a second image analysis based a one-dimensional power spectraldensity of a black and white instance of the one or more captured imagesand the one or more reference images, the one-dimensional power spectraldensity derived by integrating a Discrete Fourier Transform of atwo-dimensional image over annular rings of substantially equal areastarting from the center of the two-dimensional image, the imageanalysis module computes the combined image distance measure bycombining an image distance measure generated from the color histogramsdata with an image distance measure generated from the one dimensionalpower spectral density comparison, and the image analysis module appliesadjustable weights for each of the first and second paths, wherein thecontrol unit takes an action to affect external to the image processingunit based on the combined image distance from the image comparison. 2.The image processing unit of claim 1, wherein the image processing unittakes the action, responsive to the combined image distance exceeding athreshold.
 3. The image processing unit of claim 1, wherein the imageanalysis module compares a texture from the one or more captured imagesof food against a texture from the one or more reference images of food.4. The image processing unit of claim 1, wherein the histograms of imagecolors comprise a red histogram, a green histogram, and a bluehistogram.
 5. A computer-implemented method in an image processing unit,at least partially implemented in hardware, for improving comparison ofimages by determining combined image distance from dual path analyses,the method comprising: capturing one or more images of a subject;comparing the one more captured images against one or more referenceimages with a control unit electrically coupled to the image sensor andto a storage device and comprising a processor and a memory device,wherein the step of comparing comprises: determining a combined imagedistance between the one or more captured images and the one or morereference images from a dual path analysis, the dual path analysiscomprising a first image analysis based on histograms of image colorsand a second image analysis based a one-dimensional power spectraldensity of a black and white instance of the one or more captured imagesand the one or more reference images, the one-dimensional power spectraldensity derived by integrating a Discrete Fourier Transform of atwo-dimensional image over annular rings of substantially equal areastarting from the center of the two-dimensional image, computing thecombined image distance measure by combining an image distance measuregenerated from the color histograms data with an image distance measuregenerated from the one-dimensional power spectral density comparison,and applying adjustable weights for each of the first and second paths;and taking an action to affect external to the image processing unitbased on the combined image distance from the image comparison.
 6. Acomputer-readable medium storing source code that, when executed by aprocessor, performs a computer-implemented method in an image processingunit, at least partially implemented in hardware, for improvingcomparison of images by determining combined image distance from dualpath analyses, the method comprising: capturing one or more images of asubject; comparing the one more captured images against one or morereference images with a control unit electrically coupled to the imagesensor and to a storage device and comprising the processor and a memorydevice, wherein the step of comparing comprises: determining a combinedimage distance between the one or more captured images and the one ormore reference images from a dual path analysis, the dual path analysiscomprising a first image analysis based on histograms of image colorsand a second image analysis based a one-dimensional power spectraldensity of a black and white instance of the one or more captured imagesand the one or more reference images, the one-dimensional power spectraldensity derived by integrating a Discrete Fourier Transform of atwo-dimensional image over annular rings of substantially equal areastarting from the center of the two-dimensional image, computing thecombined image distance measure by combining an image distance measuregenerated from the color histograms data with an image distance measuregenerated from the one-dimensional power spectral density comparison,and applying adjustable weights for each of the first and second paths;and taking an action to affect external to the image processing unitbased on the combined image distance from the image comparison.